The present invention relates to a color signal processing device and a color signal processing method which are applied to color digital cameras etc.
Today, color signal processing devices and color signal processing methods are widely employed in digital cameras which are used as input devices for computers. Especially, digital still cameras for shooting and recording still images are now being widely used to the same extent as the conventional cameras using AgCl films, and video meeting (teleconference) systems using digital cameras and computers are also being used. Also in the field of video cameras for recording moving pictures in magnetic tapes, digital video cameras are prevailing since image deterioration does not occur due to image processing. For processing image signals in such digital cameras, a special-purpose image processing logic circuit or a general-purpose CPU (and software) is utilized.
FIG. 1 is a block diagram showing an example of a conventional color signal processing device and a conventional color signal processing method for processing color signals outputted by a solid-state image pickup device. The conventional color signal processing device and the conventional color signal processing method are employed in video cameras for recording moving video pictures or digital still cameras for recording still images.
In the following, an example which employs a color filter shown in FIG. 3 on a solid-state image pickup device (CCD) 200 will be explained, for instance. Referring to FIG. 3, each area on the color filter with a letter xe2x80x9cRxe2x80x9d, xe2x80x9cGxe2x80x9d or xe2x80x9cBxe2x80x9d corresponds to a pixel (element) of the solid-state image pickup device 200. The area on the color filter with a letter xe2x80x9cRxe2x80x9d transmits red light and filters out green light and blue light. The area with a letter xe2x80x9cGxe2x80x9d transmits green light and filters out blue light and red light. The area with a letter xe2x80x9cBxe2x80x9d transmits blue light and filters out red light and green light. Each pixel (element) of the solid-state image pickup device 200 receives the red light, the green light or the blue light which passed the corresponding area of the color filter, and outputs an analog image output signal whose level corresponds to the intensity of the light that passed the corresponding area. The analog image output signal outputted by the solid-state image pickup device 200 is supplied to an A/D converter 201. The AID converter 201 converts the analog image output signal into a digital signal and outputs the digital signal to a color interpolation section 202. In the color interpolation section 202, digital signals corresponding to colors (wavelength regions) which the area of the color filter did not transmit (in the case of the pixel corresponding to the area with the letter xe2x80x9cRxe2x80x9d, a green signal and a blue signal) are obtained by interpolation using digital signals (green signals and blue signals) from adjacent pixels. In the case where the color space of the output of the color interpolation section 202 is the RGB color space (i.e. in the case where the color filter on the solid-state image pickup device 200 has red areas, green areas and blue areas), signals outputted by the color interpolation section 202 are composed of a red signal r0, a green signal g0 and a blue signal b0. The signals r0, g0 and b0 outputted by the color interpolation section 202 are supplied to a matrix operation section (color space conversion section) 203 having a 3xc3x973 matrix. The matrix operation section 203 executes color space conversion to each signal using the 3xc3x973 matrix so as to match a desired color space. The output of the matrix operation section 203 (a red signal r1, a green signal g1 and a blue signal b1) can be obtained by the following equation. Incidentally, in the following, matrixes and vectors are represented by capital letters, and scalar quantities are represented by small (lower-case) letters.                               (                                                    r1                                                                    g1                                                                    b1                                              )                =                                            M              ⁡                              (                                                                            r0                                                                                                  g0                                                                                                  b0                                                                      )                                      ⁢                          xe2x80x83                        ⁢            M                    =                      (                                                            m11                                                  m12                                                  m13                                                                              m21                                                  m22                                                  m23                                                                              m31                                                  m32                                                  m33                                                      )                                              (        1        )            
The 3xc3x973 matrix M includes 9 color separation coefficients m11, m12, m13, m21, m22, m23, m31, m32 and m33 as its matrix elements. In the case where the color signal processing device is provided with white balance sections 204, 205 and 206, diagonal elements m11, m22 and m33 of the matrix M is set at 1, and thus the matrix M substantially has 6 unique elements. If the color signal processing device is not provided with the white balance sections 204, 205 and 206, the matrix M has 9 unique elements.
The output of the matrix operation section 203 (the red signal r1, the green signal g1 and the blue signal b1) is transformed by the white balance sections 204, 205 and 206 as:                               (                                                    r2                                                                    g2                                                                    b2                                              )                =                  WB          ·                      (                                                            r1                                                                              g1                                                                              b1                                                      )                                              (        2        )            
where WB is a 3xc3x973 diagonal matrix having three diagonal elements (white balance coefficients) wbr, wbg and wbb.
Subsequently, the red signal r2, the green signal g2 and the blue signal b2 outputted by the white balance sections 204, 205 and 206 are supplied to gamma correction sections 207, 208 and 209 respectively. The gamma correction sections 207, 208 and 209 execute gamma correction to the red signal r2, the green signal g2 and the blue signal b2 respectively, and the output of the gamma correction sections 207, 208 and 209 (a red signal r3, a green signal g3 and a blue signal b3) is supplied to an output device 210.
The conventional color signal processing device and the conventional color signal processing method described above involve the following drawbacks or problems.
First, in the case where a general-purpose CPU is employed for color image signal processing, a digital camera for recording still images is necessitated to take a long time for executing and completing the color signal processing since the moment the shutter button is pushed. In digital cameras for recording moving pictures, the number of frames can not be set large in comparison with the case where a special-purpose image processing logic circuit is employed for color image signal processing, therefore, it is difficult to obtain enough smoothness of the moving picture. Such problems occur due to processing speed of the general-purpose CPU.
Second, in the case where a special-purpose image processing logic circuit is employed for color image signal processing, the logic circuit has to include a large number of multiplication circuits. Therefore, the number of gates (i.e. the circuit scale) of the special-purpose image processing logic circuit in the digital camera for still images or in the digital camera for moving pictures is necessitated to be large, and thus power consumption of the digital camera is necessitated to be large. Especially in the case of a portable digital camera which is powered by a battery pack or batteries, continuous use (operation) time of the digital camera is necessitated to be short.
The above problems occur since the number of multiplications required in the conventional color signal processing device and the conventional color signal processing method is large (9 multiplications per one pixel in the equations (1) and (2)) and thus the conventional device and method are not suitable for real-time color image signal processing. Even in the case where the color image signal processing is executed by a special-purpose image processing logic circuit, 9 multiplication circuits having large number of gates become necessary, and thus it is difficult to reduce the power consumption of the digital camera.
It is therefore the primary object of the present invention to provide a color signal processing device, by which necessary circuit scale and power consumption of the logic circuit can be reduced.
Another object of the present invention is to provide a color signal processing device, by which color image signal processing with increased processing speed can be realized.
Another object of the present invention is to provide a color signal processing method, by which necessary circuit scale and power consumption of the logic circuit can be reduced.
Another object of the present invention is to provide a color signal processing method, by which color image signal processing with increased processing speed can be realized.
In accordance with a first aspect of the present invention, there is provided a color signal processing device for processing color signals outputted by an image pickup device provided with a color filter having color areas of N colors (Nxe2x89xa73) corresponding to each pixel of the image pickup device, comprising a vector operation means for executing color space conversion. The vector operation means is supplied with a color signal which corresponds to the intensity of light that reached a pixel of the image pickup device through a color area of the color filter on the pixel, obtains a color signal component vector by multiplying the color signal by a vector having N vector elements, and thereby extracts N color signal components corresponding to the N colors from the color signal.
In accordance with a second aspect of the present invention, in the first aspect, the N color signal components extracted by the vector operation means are composed of a red signal component, a green signal component and a blue signal component.
In accordance with a third aspect of the present invention, in the first aspect, the N color signal components extracted by the vector operation means are composed of a magenta signal component, a cyan signal component, a green signal component and a yellow signal component.
In accordance with a fourth aspect of the present invention, in the first aspect, the color signal processing device further comprises a color interpolation means. The color interpolation means is placed after the vector operation means and executes vector interpolation using the color signal component vector of the pixel and color signal component vectors of adjacent pixels.
In accordance with a fifth aspect of the present invention, in the fourth aspect, the color signal processing device further comprises white balance means which are placed between the vector operation means and the color interpolation means.
In accordance with a sixth aspect of the present invention, in the fourth aspect, the color signal processing device further comprises white balance means which are placed after the color interpolation means.
In accordance with a seventh aspect of the present invention, in the first aspect, the color signal processing device comprises a CPU (Central Processing Unit) that is capable of executing multimedia instructions each of which makes the CPU execute a plurality of similar operations by one instruction, and the CPU capable of executing multimedia instructions executes the operation of the vector operation means.
In accordance with an eighth aspect of the present invention, in the seventh aspect, the multimedia instructions executed by the CPU are SIMD (Single Instruction Multiple Data) instructions.
In accordance with a ninth aspect of the present invention, in the fourth aspect, the color signal processing device comprises a CPU (Central Processing Unit) that is capable of executing multimedia instructions each of which makes the CPU execute a plurality of similar operations by one instruction, and the CPU capable of executing multimedia instructions executes the operation of the color interpolation means.
In accordance with a tenth aspect of the present invention, in the ninth aspect, the multimedia instructions executed by the CPU are SIMD (Single Instruction Multiple Data) instructions.
In accordance with an eleventh aspect of the present invention, there is provided a color signal processing method for processing color signals outputted by an image pickup device provided with a color filter having color areas of N colors (Nxe2x89xa73) corresponding to each pixel of the image pickup device. The color signal processing method comprises a color signal input step and a vector operation step. In the color signal input step, a vector operation means for executing color space conversion is supplied with a color signal which corresponds to the intensity of light that reached a pixel of the image pickup device through a color area of the color filter on the pixel. In the vector operation step, the vector operation means obtains a color signal component vector by multiplying the color signal by a vector having N vector elements and thereby extracts N color signal components corresponding to the N colors from the color signal.
In accordance with a twelfth aspect of the present invention, in the eleventh aspect, the N color signal components extracted in the vector operation step are composed of a red signal component, a green signal component and a blue signal component.
In accordance with a thirteenth aspect of the present invention, in the eleventh aspect, the N color signal components extracted in the vector operation step are composed of a magenta signal component, a cyan signal component, a green signal component and a yellow signal component.
In accordance with a fourteenth aspect of the present invention, in the eleventh aspect, the color signal processing method further comprises a color interpolation step after the vector operation step. In the color interpolation step, vector interpolation is executed using the color signal component vector of the pixel and color signal component vectors of adjacent pixels.
In accordance with a fifteenth aspect of the present invention, in the fourteenth aspect, the color signal processing method further comprises a white balance step between the vector operation step and the color interpolation step.
In accordance with a sixteenth aspect of the present invention, in the fourteenth aspect, the color signal processing method further comprises a white balance step after the color interpolation step.
In accordance with a seventeenth aspect of the present invention, in the eleventh aspect, the vector operation step is executed by a CPU (Central Processing Unit) that is capable of executing multimedia instructions each of which makes the CPU execute a plurality of similar operations by one instruction.
In accordance with an eighteenth aspect of the present invention, in the seventeenth aspect, the multimedia instructions executed by the CPU are SIMD (Single Instruction Multiple Data) instructions.
In accordance with a nineteenth aspect of the present invention, in the fourteenth aspect, the color interpolation step is executed by a CPU (Central Processing Unit) that is capable of executing multimedia instructions each of which makes the CPU execute a plurality of similar operations by one instruction.
In accordance with a twentieth aspect of the present invention, in the nineteenth aspect, the multimedia instructions executed by the CPU are SIMD (Single Instruction Multiple Data) instructions.